1. Technical Field
This invention relates to feature extraction and pattern recognition devices, and in particular, to a neural network signal processor that can internally develop, or "learn" the algorithms required for identification of features directly from input sensor signals.
2. Discussion
The ability to recognize patterns is a major step towards the development of artificial systems that are capable of performing perceptual tasks that currently only biological systems can perform. Speech and visual pattern recognition are two areas in which conventional computers are seriously deficient. In an effort to develop artificial systems that can perform these and other tasks, a number of signal processing techniques have been developed to extract features from signals. These techniques typically involve extensive preprocessing. Such preprocessing may require, for example, measuring pulse width, amplitude, rise and fall times, frequency, etc. Once these features are extracted they can be matched with stored patterns for classification and identification of the signal. The software required to accomplish these steps is often complex and is time consuming to develop. Moreover, conventional digital signal processors are not able to tolerate certain variations in the input signal, such as changes in orientation of a visual pattern, or differences in speakers, in the case of speech recognition.
In recent years it has been realized that conventional Von Neumann computers, which operate serially, bear little resemblance to the parallel processing that takes place in biological systems such as the brain. It is not surprising, therefore, that conventional signal processing techniques should fail to adequately perform the tasks involved in human perception. Consequently, new methods based on neural models of the brain are being developed to perform perceptual tasks. These systems are known variously as neural networks, neuromorphic systems, learning machines, parallel distributed processors, self-organizing systems, or adaptive logic systems. Whatever the name, these models utilize numerous nonlinear computational elements operating in parallel and arranged in patterns reminiscent of biological neural networks. Each computational element or "neuron" is connected via weights or "synapses" that typically are adapted during training to improve performance. Thus, these systems exhibit self-learning by changing their synaptic weights until the correct output is achieved in response to a particular input. Once trained, neural nets are capable of recognizing a target and producing a desired output even where the input is incomplete or hidden in background noise. Also, neural nets exhibit greater robustness, or fault tolerance, than Von Neumann sequential computers because there are many more processing nodes, each with primarily local connections. Damage to a few nodes or links need not impair overall performance significantly.
There is a wide variety of neural net models utilizing various topologies, neuron characteristics, and training or learning rules. Learning rules specify an internal set of weights and indicate how weights should be adapted during use, or training, to improve performance. By way of illustration, some of these neural net models include the Perceptron, described in U.S. Pat. No. 3,287,649 issued to F. Rosenblatt; the Hopfield Net, described in U.S. Pat. Nos. 4,660,166 and 4,719,591 issued to J. Hopfield; the Hamming Net and Kohohonen self-organizing maps, described in R. Lippman, "An Introduction to Computing with Neural Nets", IEEE ASSP Magazine, April 1987, pages 4-22; and the Generalized Delta Rule for Multilayered Perceptrons, described in Rumelhart, Hinton, and Williams, "Learning Internal Representations by Error Propagation", in D. E. Rumelhart and J. L. McClelland (Eds.), Parallel Distributed Processing; Explorations in the Microstructure of Cognition. Vol. 1: Foundations. MIT Press (1986).
While each of these models achieves varying degrees of success at the particular perceptual tasks to which it is are best suited, the parallel inputs required by these systems are thought to necessitate special purpose preprocessors for real time hardware implementations. (See the above-mentioned article by R. Lippman.) For example, in Rosenblatt's Perceptron, (U.S. Pat. No. 3,287,649) each input receives a separate frequency band of an analog audio signal. More recent physiologically based preprocessor algorithms for speech recognition attempt to provide information similar to that available on the auditory nerve. Thus, while neural nets offer the distinct advantage of self-learning and elimination of software, a certain amount of preprocessing has still been required in prior neural nets. While prior neural nets may require less preprocessing than digital signal processors, they still share this disadvantage with conventional digital signal processors. Accordingly, it would be desirable to provide a neural network capable of real time signal processing that eliminates, or significantly reduces the preprocessing required.